Correction and Extension 1 Running head: CORRECTION AND EXTENSION OF WAINER (1976) Correction and Extension of Wainer's Estimating Coefficients in Linear Models: It Don't Make No Nevermind

In a citation classic, Wainer (1976) claimed that the expected loss in predictable variation due to using equal linear regression weights (applied to standardized predictors), instead of using optimal weights, is linearly increasing in k, the number of predictors. Wainer claimed to prove this while assuming that weights are independent and follow a uniform distribution on the interval (1=4;3=4). Wainer erred twice: (1) he underestimated the loss by a factor of two (Laughlin, 1978), and (2) his posited weight distribution is infeasible for k 2. When these errors are corrected, then for uniform (or symmetric beta distributed) optimal regession weights, one finds that the expected loss is nonincreasing as k increases the opposite of Wainer's result. I go on to examine the more general situation where regressors are correlated, and evaluate the expected loss by Monte Carlo integration. Only under highly implausible assumptions, about the distribution of the optimal regression weights, does one obtain expected loss notably greater than .025; this loss is nearly independent of k. Correction and Extension 3 Correction and Extension of Wainer's Estimating Coefficients in Linear Models: It Don't Make No Nevermind In a classic article, Wainer (1976) advanced two conclusions. First, he argued that the loss of predictable variation of a random variable, caused by using equal weights for regressors instead of optimal, possibly unequal weights, is likely to be small. He offered a putative proof of this, involving a second claim that the expected loss was at most k=96 (where k is the number of regressors), though he did not show all steps of his proof. Laughlin (1978) pointed out an error of algebra, and argued that the expected loss is really k=48, twice what Wainer claimed. The purpose of this note is to show that Wainer's proof is further in error whenever k 2. In fact, the relationship between loss and k depends on the correlations among the predictors. Nonetheless, plausibility considerations do establish that the loss of predictability from using equal weights is indeed generally small. To demonstrate these points, the article proceeds in four steps. First, Wainer's proof is corrected (following Laughlin, 1978), to obtain the correct general expression for the expected loss, requiring the existence of two finite moments and independent predictors. In the second step, it is shown that Wainer's Correction and Extension 4 postulated uniform distribution of optimal weights (adopted by Laughlin, 1978, in his correction of Wainer) is infeasible for any k 2. Third, the expected loss is derived for a class of feasible weight (and predictor intercorrelation) distributions: the symmetric beta distribution family. Finally, the results of Wainer are extended to describe the situation obtained when predictors are correlated. Analytic and numerical evaluations of expected loss are tabled for non-negative predictor intercorrelations, and uniformly distributed weights. These give upper bounds on the expected loss when the weight distribution is more concentrated than the uniform, e.g., all unimodal distributions. Pruzek and Frederick (1978) argued that, owing to the restrictiveness of assumptions made by Wainer, his conclusions misrepresent the comparative efficiencies of equal vs. least-squares regression weights over important parts of the parameter space. They also related the least-squares weights to the predictor intercorrelations and the predictor-criterion correlations via a useful principal components-type reparameterization. Many other authors have weighed in (pun intended) with data and opinions about the general position advocated by Wainer: both in favor of it (Bloch & Moses, 1988; Burt, 1950; Dawes, 1979; Dawes & Corrigan, 1974; Dorans & Drasgow, 1978; Einhorn & Hogarth, 1975; Gulliksen, 1950; Raju, Reyhan, Edwards, & Fleer, 1990; Richardson, 1941; Schmidt, Johnson, Correction and Extension 5 & Gugel, 1978; Tukey, 1948) and against it (Keren & Newman, 1980; Stillwell, Seaver, & Edwards, 1981) Wainer's general position. (Cattin, 1978, discussed a case-by-case method for choosing between optimal and equal weights.) However, none of these authors directly addressed the question Wainer asked and tried, however imperfectly, to answer: By how much do optimal (least squares) regression weights outperform equal weights, across a relevant parameter space? For example, Keren and Newman (1980) point out (correctly) that when predictors are negatively uncorrelated (rather than uncorrelated, as assumed by Wainer), as can occur when there are suppressor variables (Horst, 1941), least squares can greatly outperform equal weights. However, these authors did not give empirical data to help us understand how often we can expect to encounter suppressor variables in the real world; and they did not quantify the size of the aggregate loss due to using equal weights when predictors are negatively correlated, or the extent to which this loss would be vitiated by small losses when predictors are independent; let alone quantify relative performance when predictors are positively intercorrelated. The question of aggregate loss may seem at first an uninteresting one. Why not compare them on a case by case basis, and eschew averaging over situations which are anywhere from very slightly to very greatly different? The answer is in Correction and Extension 6 two parts. First, the expected comparative efficiencies of least squares vs. equal weights depend on population parameters, not sample estimates of them. A caseby-case approach is only partially enlightened by sample estimates of the relevant parameters and the light will be dim when one has small samples, as one often does in psychology. It is useful in such situations to have a valid generalization, about the loss due to using equal weights, for a particular prediction domain. This generalization acts somewhat like a Bayesian prior probability, in serving as an anchor for samplebased adjustments to one's opinion about whether to prefer equal or least squares weights. Second, a population average does not lose all its value when the population is known to be heterogeneous. Social statistics rely heavily on this fact the mean (or median) income is a useful statistic even though we know it varies with the values of many factors such as education and age. If the average loss, due to using equal weights, is sufficiently small over a relevant part of the entire parameter space, then this justifies the conclusion that the loss is not very large over very much of the space. This paper sticks with Wainer's original question about the size of aggregate (expected) loss in predictive efficiency, due to using equal instead of optimal least squares weights in linear regression. It corrects the error of Wainer (1976) previously pointed out by Laughlin (1978) (and admitted by Wainer, 1978), but Correction and Extension 7 goes on to correct Laughlin's own error in relying on a mathematically impossible assumption. It extends Wainer's result by considering relative predictive efficiency of the two types of weights, when predictors are positively intercorrelated. Moreover, there is a particular focus on the behavior of the expected loss, due to using equal weights, as the number of predictors increases. Recapitulation of Wainer's Argument Suppose that a random variable y, assumed without loss of generality to have zero mean and unit variance, is to be predicted by the linear regression equation y = b y + e = X + e (1) where items in boldface upper case are matrices, while boldface lower case indicates a vector; b X is the best linear unbiased estimator of X; X is an n k matrix of regressors (assumed here to be of full column rank). We assume, without loss of generality, that elements of the columns of X are sampled from populations with zero means and unit variances. e is a vector of independent, identically distributed residuals, with mean = 0 and variance 2 , and is optimal in the least squares sense. It is well known that = 1 , where is the matrix of correlations Correction and Extension 8 among the regressors and is the vector of zero-order predictor-criterion Pearson correlations. If predictors are uncorrelated, the optimal weights are just = . By contrast, one could also predict y via weights i = j ; i; j = 1; : : : ; k, that is, all equal weights. The linear model is y = e y + = aX1 + ; (2) where 1 is a vector of ones. The value of a that minimizes the sum of squared errors of prediction, 0 , is given by the solution of